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SYSTEM FOR GUIDING A MEDICAL INSTRUMENT IN A PATIENT BODY 

EIEIiD OF THE INVENTION 

The present invention relates to a medical system con]^rising a medical instrument to 
5 be guided in a patient body and means for visualizing said medical instrument. The invention 
also relates to a method to be used in said system. The invention finds for &xzttsplc its 
application for guiding a catheter inside the heart of a patient during an electrophysiology 
interventional procedure. 

10 BACKGROUND OF THE INVENTION 

Clinical ^plications in which a medical instrumrat has to be guided into the body of 
a patient are becoming widespread. Notably tiie growing interest in minimal- invasive 
methods for the treatment of cardiac diseases necessitates the development of methods and 
devices allowing the physician to guide a medical instrument to predetemiined positions 

15 inside or outside the heart In electrophysiology for exanqple, it is necessary to guide a 

catheter to a plurality of predetermined positions on the ventricular or atrial walls in order to 
measure an electrical pulse or to bum wall tissues. 

US patent 6,587^709 discloses a system for guiding a medical instrument in the body 
of a patient Such a system acquires a live 3D ultrasound image data set using an ultrasound 

20 probe. An advantage of acquiring a 3D image data set is to get depth itLformation. An 

advantage of using a live 3D ultrasound image modality is that the surrounding anatomy is 
visible, which facilitates the guidance of the medical instrument by the physician. The system 
fialher comprises localization means for localizing the medical instrument within the 3D 
ultrasotmd data set» which locates three ultrasound receivers mounted on the medical 

25 instrument relatively to said ultrasound probe. Such a localization allows an automatic 
selection of a plane to be imaged^ which comprises at least a section of the medical 
instrument Therefore no readjustment of the ultrasound probe position by hand is necessary. 

A first drawback of such a 3D ultrasound data set is that it has a narrow viewing fields 
which does not cover the whole part of the patient body concerned by a catheter, introduction 

30 and placement Therefore, for guiding the catheter during the whole procedure, the 

ultrasound probe has to be moved several times. At each displacement, a pre-operative step 
of locating the ultrasoxmd probe in a referential of the interventional room is needed, because 
the location of tiie catheter is measured relatively to the ultrasound probe location. Such a 
pre-operative step may delay and compUcate the interventional procedure. 
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A second drawback of the ultrasound imaging modality is fhat it has a low resolution 
Therefore, flie acquired 3D ultrasound data set does not give an image of the catheter and its 
surrounding of acceptable quality. 

A thbxi drawback of the ultrasound imaging modality is that there are some zones of 
5 the patient body where the thoracic cage blocks the ultrasoxmd scan and no exploitable image 
can be output 



SUMMARY OF THE INVENTION 

The object of the invention is therefore to provide a medical system for guiding a 
1 0 medical instrument in a pati^t body, which gives an improved visibility of the medical 
instrument and its surrounding anatomy during the whole procedure. 
This is achieved by a medical system comprising: 

- a medical instrument to be guided in apatient bod^r, 

- X-Ray acquisition meaiis for acquiring a two-dimensional X-ray image of said medical 
IS instrument, 

- ultrasound acquisition means for acquiring a three-dimensional ultrasoimd data set of said 
medical instrument using an ultrasound probe, 

- means for providing a localization of said ultrasound probe within a referential of said X- 
ray acquisition means, 

20 - means for selectmg a region of interest around said medical iiistrument in the 3D 
ultrasound data set, that define a first localization of said region of interest within a 
referential of said ultrasoimd acquisition means, 

- means for converting said first localization of said region of interest within said 
referential of the ultrasound acquisition means into a second localization of said region of 

25 interest within said ref^ntial of the X-ray acquisition means, using said localization of 

the ultrasound probe, 

- means for generating and displaying a bi- modal representation of said medical instrument 
in which said two-dimensional X-rsQr image and the three-dimensional ultrasound data 
included in said region of interest are combined using said second localizatioiL 

30 

With the invention^ a bimodal representation is provided, in which two-dimensional 
(2D) X-Ray data and three-dimensional (3D) liltrasound data are combined 2D X-ray data 
provide a good visibilily and a high resolution of the medical instrument and of bone 
structures. 2D X-Ray data also benefit from a large viewing field, which allows a 
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visualization of the whole area of fhe patient body concerned by the electcophysiology 
procedure. 

3D ultrasound data provide a good visibility of soft tissues and sometimes 
vascularities in a surrounding of the medical instrument In addition, 3D ultrasound data give 
5 an indication of depth, which is not provided by the 2D X-Ray image, because said X-Ray 
image only provides a projection of said medical instrument in accordance with a geometry 
of the X-Ray acquisition means. Such a geometry defines lines of projection, along which 
absorptions of X-rays by the e3q)osed tissues of the patient are accumulated. Therefore, the 
visibility of the surrounding of the medical instrument is improved by the combination of the 

10 2D X-Ray and the 3D ultrasound data. 

In order to provide such a combiuation, the system firstly localizes tbe ultrasoimd 
probe and the 3D ultrasoimd data set in a referential of the X-Ray acquisition means. Such a 
referential of the X-ray acquisition means is supposed to be fixed. Therefore, assuming that 
the ultrasound piobe does not move, a position of any point of the 3D tiltrasound data set in 

15 said referential the X-ray acquisition means can be deduced. As the position of the 2D X-Ray 
image on the X-Ray detector within the referential of the X-Ray acquisition means is given 
by the geometry of the X-Ray acquisition means, the localization of the 3D ultrasound data 
set within the referential of the X-Ray acquisition means allows a mapping of the 3D 
ultrasound data set with the 2D X-Ray image, L e. a mapping of a projection of any point of 

20 the 3D ultrasound data set in accordance with the geometry of the XRay acquisition means . 
with a point of the 2D X-Ray image. 

The system in accordance with the invention further selects a region of interest 
surrounding the medical instrument in the 3D ultrasoimd data set and provides a first 
localization of said region of interest within a referential of the ultrasound acquisition means. 

25 Such a selection, which is made either manually or automatically, aims at suppressing any 
ultrasound data which could occlude the visuaUzation of the medical instrument. 

The first localization, which is expressed with coordinates of a referential of the 3D 
ultrasound acquisition means, is then converted into a second localization of the medical 
instrument within the referential of the X-Ray acquisition means, using the localization of the 

30 ultrasound probe. 

The system in accordance with the invention finally generates a bimodal 
representation, in which the 3D tdtrasoimd data included in said region of interest are 
combined to the 2D X-Ray data using said second localization of the region of interest within 
the referential of the X-Ray acquisition means. 
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Preferably, the bimodal representation is generated on the basis of the 2I>X-iay 
hnage. In this 2D X-ray image all the X-ray intensity values of points having corresponding 
points in the selected region of interest of the 3D ultrasotmd data set are combined by the 
ultrasound intensity values. 
5 Said means for selecting a region of interest are intended to define a reference plane 

in which a part of said medical instrument is included. In a first embodiment of the invention 
the region of interest is included into said reference plane which^ for instance, comprises the 
extremity of the medical instrument which is in contact with the wall tissue and is 
perpendicular to the orientation of the X-Ray acquisition means. Therefore, the means for 

10 generating a bimodal representation are intended to combine the 2D X-Kay image with the 
2D ultrasound image which is obtained by sanq>ling the 3D ultrasound data set over the 
reference plane coordinates in the referential of the X-Ray acquisition means. A first 
advantage of the first embodiment of the invention is that it is very simple. A second 
advantage is that any ultrasound data yMoh could occlude the medical instrument and its 

1 5 surrounding have been removed. 

In an alternative, the selection means comprises detection means for detecting the 
medical instrument in the region of interest of the 3D ultrasound data set Such a detection is 
for instance achieved automatically by using image processing techniques, for instance a 
filter for enhancing and thresholding elongated shapes. In the bimodal representation the X- 

20 ray intensity values of the points of the 2D X-ray image belonging to the detected medical 
instrument ate advantageously kept unchanged. A first advantage is that the bimodal 
representation benefits fix>m the high resolution of the medical instrument provided by the X- 
ray acquisition means. 

A second advantage of this detection is that it is based on image processmg 

25 techniques and does not require any specific medical instrument, like a medical instrument 
equipped with active localize. Considering that the medical instrument has to be changed 
for each new patient; another advantage of the system in accordance with the invention is to 
allow a non negligible cost savings. 

A third advantage of this detection is that it gives a localization of the ejctremity of the 

30 medical instrument This localization, combined with the bimodal representation, may help 
generating of an electrical activation map of the heart cavity wall. As a matter of fact, in such 
a procedure, the medical instrument is a catheter equipped at its extremity with a sensor for 
measuring electrical pulses on a heart cavity wall. The user activates the sensor when the 
catheter is in contact vdth the heart cavity wall. A measurement of the electrical pulse at the 
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current position of the catheter is made.The localization of the catheter provided by the 
syst^n in accordance -wifli the mvention provides a location of the point corresponding to 
said current position and said electrical measurement in the electrical activation map. By 
visuali2dng the bimodal representation the user has the possibility to evaluate a distance 
5 between the current measurement point and previous measurement points. Therefore, the 

system in accordance witti the mvention facilitates a quicks uniform and complete mapping of 
the cavity wall. 

In a second embodiment of the invention, the system in accordance with the invention 
further comprises means for segmenting a wall tissue region in the 3D ultrasoxmd data set. 

10 Therefore, in the 2D X-ray image, only the X-ray intensity values of the points belonging to 
said wall tissue region are combined with the corresponding ultrasoimd intensity values. 
Therefore, the bimodal representation overweights X-Ray intensity values inside the cavity 
wall and ultrasound intensity values in and outside the cavity wall. An advantage is that the 
bimodal representation locally benefits firom the best information available. 

15 In a third embodiment of the invention, the region of interest is a 3D subset of the 3D 

ultrasoxmd data set, which either lies behind the reference plane along the X-Ray direction or 
forms a slab around the reference plane. A volimie rendered view of the 3D ultrasound data 
included in said selected region of interest is provided. The bimodal representation is built up 
by combining the intensity values of points of the 2D X-ray projection with the intensity 

20 values of the corresponding points in the volume rendered view. An advantage is to provide a 
perspective view of the surrounding tissues. 

The system in accordance with the invention is capable of acquiring the 2D X-Ray 
image and the 3D ultrasound data set in reattime. Therefore, the medical instrument can be 
tracked in each new 2D XRay image and 3D ultrasound data set. 

25 In a fourth embodiment of the invention, the system comprises controUiag means for 

periodically triggering the probe localization means. As a matter of fact, the position of the 
ultrasomd probe within the referential of the X-Ray acquisition means may be changed by 
external movements of the patient like respiratory movements during the clinical procedure. 
Therefore, the localization of the ultrasound probe has to be updated regularly. 

30 In a fifth embodiment of the invention, the system comprises means for estimating 

and compensating a motion of a cim^nt 3D idtrasoimd data set acquired at a time t with 
respect to a previous 3D ultrasound data set acquired at a time t- 1 . An advantage is that small 
displacements of the ultrasound probe can be corrected without triggering the probe 
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localization means and fheiefore witbiout inteinqpting the live visualisation of the medical 
instrument. 

These and other aspects of the invention will be ^parent from and will be elucidated 
with reference to the embodiments described hereinafter. 



BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will now be described in more detail^ by way of example, with 
reference to the accompanying drawings, wherein: 
10 - Fig. 1 is a schematical drawing of a system in accordance with the invention, 

- Fig. 2 is a schematical drawing of means for localizing the ultrasound piobe within the X- 
lay refer^tial, when the ultrasound probe is equipped with active localize, 

- Figs. 3, 4a and 4b are schematical drawings of means for localizing the ultrasound probe 
within the X-ray referential, when the probe is equq)ped with a belt comprismg radio- 

1 5 opaque markers, 

- Fig. 5 is a schematical drawing of means for localizing the medical instrument and 
detomining a plane comprising said medical instrument within Ihe 3D ultrasound data set 
in accordance with the invention, 

- Fig. 6 is a schematical drawing of the converting means for converting a first localization 
20 of the region of interest within the referential of the ultrasound acquisition means into a 

second localization of the region of interest within the reierential of the X-Ray 
acquisition means, 

- Fig. 7 is a schematical drawing of means for generating a bimodal representation in 
accordance with a first embodiment of the invention, 

25 - Fig. 8 is a schematical drawing of means for generating a bimodal representation in 
accordance with a second embodiment of the invention, 

- Fig. 9 is a schematical drawing of means for generating a bimodal representation in 
accordance with a third embodiment of the invention, 

- Fig. 10 is a schematical drawing of means for compensating a motion between a current 
30 3D ultrasound data set acquired at a currrat time and a previous 3D ultrasoimd data set 

acquired at a previous time, 

- Fig. 1 1 is a functional diagram of the method in accordance with the invention. 



DETAILED DESCRBPTION OP THE INVENTION 
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The present invention relates to a system for guiding a medical instrument in a patient 
bocfy. Such a system is particularly adapted for guiding a catheter within the heart cavities in 
order to diagnose and cure heart diseases, but it can more generally be used for guiding any 
other medical instrument in the patient body, like for instance a needle. 
5 The scheniatical drawing of Fig, 1 shows a patient 1 , who is arranged on a patient 

table 2 and whose symbolically indicated heart 3 is subjected to a treatment by means of a 
catheter 4 introduced into the body. The system conq)rises means 5 for acquiring a 2D X-iay 
image of the patient body. Said X-ray acquisition means comprise a focal X-ray source 6 and 
a detector 7, Advantageously, these X-ray acquisition means 5 are a C-arm system, as it is 

10 usually the case in a cathlab room. An advantage of such a C-arm system is to be able to 
have a rotational motion aroimd the patient body in order to produce a plurality of 2D Xray 
images of the patient at known orientation angles. 

The system in accordance with the mvention further comprises means 8 for acquiring 
a 3D ultrasound data set fix3m an ultrasound probe 9, which has been placed on the patient 

15 body and fixed by fixation means, for instance a belt 10 or a stereotactic arm. It should be 
noted that both 2D X-ray image and 3D ultrasound data set are acquired in reattime, which 
enables a live visualization of the medical instrument, when it is guided inside the patient 
body. 

The X-ray acquisition means 5 are associated with a referentied of coordinates (O, x, . 

20 y, z), called X-Ray referential hereinafter, in which the geometry of the focal X-ray source 6 
and the detector 7 is known. It should be noted that the X-Ray referential (O, x, y, z) is bound 
to the fixed part of the X-Ray acquisition means and not to the C-arm. Therefore, the 
orientation of the C-arm can be e^ressed in said X-Ray referential. However, the geometry 
of the X-Ray acquisition means is dependent on a particular position of the C-arm. 

25 The system in accordance with the invention fiirther conq>rises means 1 1 for 

. . localizmg the ultrasound probe 9 witihdn the X-Ray referential (O, x, y, z), means 12 for 

selecting a region of interest around the medical instrument in the 3D ultrasound data set and 
for providing a first localization Loci of the region of interest within a referential (O*, x% y*, 
z') of the ultrasound acquisition means, means 13 for converting the first localization Loci of 

30 the region of interest within a referential (O*, x', y% z') into a second localization L0C2 of the 
region of interest within the X-Ray referential (O, x, y, z) and means for generating a 
bimodal representation BI by combining data &om the 2D X-ray image with the 3D 
ultrasound data included in the region of interest and localized by the second localization 
L0C2. The bimodal representation BI is displayed on a screen 15. 
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Referring to Fig. 2, the probe localization means 11 are, in a first approach, based on 
an active localizer 16, well-known to those skilled in tiie art, whidb is arranged on the 
ultrasound probe 9. Said active localizer 16, for instance an RF coil, is intended to ttansniit 
an RF signal to an RF receiving unit 1 7 placed under the patient body and for instance 
5 integrated into the table. The RF receiving unit transmits the received signal to measuring 
means 18 for measuring a position of the ultrasound probe 9 in a known referential, for 
instance tihe referential (O, x, y, z) of the X-ray acquisition means 5, It should be noted that 
the active localizer 16 must be two-dimensional and placed on the ultrasound probe 9 in such 
a way that a precise measurCTient of the position and of the orientation of the ultrasound 

10 probe can be calculated. It should be noted tibiat a led-based optical localizer could be used as 
well. A first advantage of this first sqpproach is that it provides a precise localisation of the 
ultrasound probe. A second advantage is that it is performed in reattime and therefore, can 
be triggered during the clinical procedure. 

In a second approach of the probe localization means 1 1 shown in Fig. 3, the 

15 ultrasound probe 9 is fixed around the body of the patient 1 with a belt 10 equipped with at 
least three non aligned interd€i)endOTt radio-opaque markers Mi, Ma and M3. For instance, 
the belt 10 comprises a plexigias plaque, in which the fliree non aligned interdependent radio- 
opaque markers are fixed. 

The three markers Mi , M2 and M3 belong to a same plane, therefore at least one 2D 

20 X-ray projection 2DXRi acquked with an orientation angle 9i of the C-arm system 5 is 
needed in order to deteimine the position of the ultrasound probe in the X-Ray referential 
(O, X, y, z). However, since the three markers are interdependent, and noibaligned, vrfiich 
means that they form a rigid tetraedre, it is welHcnown to those skilled in the art that the 
position of the probe is conq)letely specified by the X-ray projection 2DXRi . 

25 Refening to Fig. 4a, we consider a detector referential (dO, dx, dy). Six parameters 

like for instance the coordinates (dxi, dyi), (djgj, dyx), (dj^, dj^) of die projections Pi, P2, P3 
of the three markers Mi, M2 and M3 in the first 2D X-ray image 2DXR| do completely 
specify the position of the ultrasound probe 9 in the X-Ray referential (O, x, y, z). A first 
advantage of using radio-opaque markers Mi , M2 and M3 is that they appear in a 2D X-Ray 

30 projection with a very high contrast which facilitates their localization. Such a localization 
may be achieved manually or automatically. In the manual case, a user may click on at least 
two radio-opaque maxk^ in each 2D X-ray image. In the autojtnatical case, image processing 
techniques well known to those skilled in the art, like for instance a morphological filter, may 
be used for detecting the radio-opaque markers, which appear as highly contrasted blobs in 
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the 2D X-ray projections. A second advantage is that such a localization is performed in real- 
time and therefore does not imply any peri- interventional step. It should be noted however 
that there is a priori no need to displace the ultrasound probe during the clinical procedure, 
because ultrasoimd data are intended to be used once the medical instrum^t has been guided 
5 in a cavity of the patient bocfy to be investigated A third advantage is that it does not 

introduce metaUic objects in the field of view of the X-Ray and ultrasound acquisition means. 

In an alternative to the second approach^ a second 2D X-Ray image 2DXR2 is 
acquired with a second orientation angle 82 of the C-aim system 5, as shown in Fig. 4B. This 
second X-Ray image allows to determine the coordinates (dx*i, dy*i), (dx'a, dy*2)> (dx'3, 

10 dy'3) of a new set of projections P'l, P*2, P*3 of the three markers Mi, Nfc and M3. It shoxUd be 
noted that the localized points Pi , P2, P3 and P'l, P*2 and P*3 follow epipolar constraints: this 
means for instance that a line linking the source focal point to the point Pi appears as a 
projected line L'l in the second X-ray image 2DXI^, which conaprises P'l. A first advantage 
is that P* I has not to be searched within the whole image, but only on the projected line L' 1 ♦ 

15 A second advantage is that it gives a way of associating the points Pi, P2, P3 and P'l, P*2> P'3 
with the light markers Mi , M2 and M3. 

Such a localization of the ultrasoimd probe 9, which involves two 2D X-Ray images 
is not performed in real-time and has therefore to be handled in a peii- interventional step of 
the clinical procedure. An advantage of acquiring two 2D X-Ray images is that the precision 

20 of the localization is greatly increased 

Once the iiltrasound probe 9 has been located in the X-Ray referential (O, x, y, z), an 
orientation of the probe is known and therefore, the location of the 3D ultrasound data set 22, 
also called 3D ultrasoimd cone, can be deduced This is achieved by the converting means 13 
which calculate a position of a point of said 3D ultrasoimd data set in the X-Ray referential 

25 fix)m said ultrasound probe localization. A projection of said point on the detector in 
accordance with the geometry of the X-Ray acquisition means can also be deduced 

Referring to Fig. 5, the system in accordance with the invention conq>rises means 12 
for selecting a region of interest 35 around the medical instrument 4 within the 3D ultrasound 
30 data set 21. A reference plane comprising a part of the medical instrument is defined 

Advantageously, said reference plane is chosen perpendicular to the direction of orientation 
of the X-Ray acquisition means. The region of interest is obtained by creeping a 3D 
ultrasound data subset, which lies behind said reference plane or by cropping a slab which is 
formed around said reference plane. la this way, structures which could occlude the visibility 
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of medical instrument in Hie 3D ultrasomid data set are lemoved. In a first approach, the user 
mteractively selects the region of interest 35 in the 3D ultrasound data set In a secx>nd 
approach, a location of said reference plane can be predefined, for instance at a predefined 
reference depth equal to a third of the depth of the 3D data set This predefined reference 
5 plane may be fiirther rotated for searching a viewing angle view within the 3D ultrasound 
data set fiiom which tiie medical instrument is more visible. A rotated reference plane 33 is 
obtained Advantageously said viewmg angle is ^plied to the C-arm system in order to 
optimize the 2D X-ray image. 

Advantageously the selection means 12 comprise means for detecting the medical 
1 0 instrument 4 within Ae 3D ultrasound data set 2 1 . It should be noted that a medical 

instrument usually appears with a high contrast within the 3D ultrasound data set It is for 
instance the case of an electrqphysiology catheter, which conqntises a metal tip at its 
extremity. Said tip is a small and thin segment which is very echogen and leaves a specific 
signature in the 3D xiltrasound data set Therefore, either the tip end is considered as a 
1 S punctual landmark or the whole tip is considered as an elongated lanH marlr 

Consequently, said detection means involve image processing techniques, which are 
well known to those sadlled in the ar^ for enhancmg either a highly contrasted blob or 
elongated simpe in a relatively uniform background. 

The detection means allow to automatically define the reference plane 30 by a point 
20 EPi and a normal orientation N , where the point EPi for instance corresponds to the 
detected extremity of the medical instrument, for instance the end of the tip 31, and the 
normal orientationJV' to the known orientation 32 of the X-ray source 6. 

In an alternative the reference plane 33 is defined by at least three non aligned points 
EPi, EP2 and EP3 given by the detection of the medical instrument 4. A second normal is 
25 defined, which can advantageously serve for reorienting the X-ray source 6 in order to 

optimize the X-ray acquisition with reqpect to the detected position of the medical instrument 
4. 

Referring to Fig. 6, a mapping between points included into the reference plane 30, 33 
and points included into the 2D X-ray image 40 can be calculated &om the knowledge of the 
30 geometry of the X-Ray acquisition means and of the second localization of the uhxasoimd 
probe 9 within the X-Ray referential (O, x, y, z) provided by the converting means 13. For 
instance, the point EPi is projected on a point P(EPi) in flie 2D X-Ray image 40 in 
accordance with a projection line 36, 
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The generation and display means 14 in accordance with the invention axe intended to 
generate abimodal representation of the medical instrument^ in which infomiation coining 
from both the 2D X-ray image and the 3D iiltrasoimd data set are combined. 
5 Preferably, such a combination is X-ray driven, which means that it is made on the 

basis of a 2D X-ray image 40, as shown in Fig. 7. 

In a first embodiment of the inv^tion, the region of interest of tiie medical instrument 
is included into the reference plane 30, 33. Therefore, the ultmsound information contained in 
the region of interest corresponds to a 2D ultrasound view 41, which is obtained by sampling 
1 0 the 3D ultrasound data set over the reference plane 30, 33, 

The bimodal representation is an image formed such that the intensity values of all the 
points of the 2D X-ray projection 40 which have a corresponding point in the 2D ultrasound 
view 41 are combined. Such a combination is for instance defined by a scalar fitaction f of a 
first intensity value Ii of a point of the 2D X-Ray image 40 and a second intensity value h of 
15 the corresponding point in the 2D ultrasound view 4L Such a scalar function f provides an 
intensity value I, for instance, by implementing an alpha-^blending technique, which is well- 
known to those skilled in the art, in the following way: 

If a is equal to one, then the intensity value I of the bimodal representation is equal to 
20 the first X-Ray intensity Ii . On the contrary, if a is equal to zero, the intensity value I of the 
bimodal representation is equal to the second ultrasound intensity I2, which means that the 
intensity value of the point of the 2D X-Ray image is replaced by the intensity value of the 
carresponding point of the 2D ultrasound image 41. 

The ultrasound acquisition means provide a 3D ultrasound data set which is focused 
25 on the medical instrument. V/iHi tie invention, the combination of the X-Ray and ultrasound 
intensity values has the advantage to inq^rove the visibility of the tissues surrounding the 
medical instrument 

It is well known to those skilled in the art that the projection of the medical 
instrument given by the X-ray source 7 on the detector 7 is of good quality and benefits from 
30 high resolution and contrast When a detection of the medical instrument within the region of 
interest of the 3D ultrasoimd data set has been noade available by the detection means, then a 
position of the projection of the medical instrument 4 within the 2D X-ray projection 40, that 
is in the detector referential (dO, dx, dy), can be derived fi'om the position of the medical 
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insixument in the X-Ray referential (O, x, y, z). This position is for instance a set 43 of points 
of the X-ray projection coiresponding to a set of points 42 within the 2D ultrasound view 41 . 

• Advantageously, the intensity values of flie points of the 2D X-ray projection 40 
belonging to the detected medical instrument are given to the corresponding points of the 
S bhnodal representation. An advantage is to keep the good visibility and resolution of the 
medical instrument provided by the X-ray acquisition means. 

In a second embodiment of the invention shown in Fig, 8, the system in accordance 
wiOi the invention furOier comprises means for segmenting a wall tissue r^on, for instance 
the endocardiac wall 44 ia the neigbondiood of the medical instrument 4. This is achieved by 

1 0 image processing techniques such as intensity value diresholding, since wall tissues like 
myocardium appear brighter than blood in ultrasound images. 

Another possibility is to detect borders, for instance by using an active contour 
technique (also called "snake*'). This technique, well known to those skilled in tiie art, JBrstly 
consists in defining an initial contour and secondly in making said initial contour evolve 

15 under the influence of internal and external forces. A final contour 46 is obtained. It is then 
possible to difiTerratiate points located inside fi*om points located outsdde the contour 46 and 
to only give to the outside points the intensity values of the corresponding points of the 2D 
ultrasound view 41 . An advantage of this second embodiment is to benefit firom X-ray 
information in a larger surrounding of the medical instrument 4. 

20 In a third embodiment of the invention schematically prasented in Fig. 9, the 

system in accordance with the invention finrther comprises means for generating a volume 
rendered image 51 of the defined region of interest 35. In the present exanq>le, the region of 
interest 35 is a slab 50. The volume rendered image 51 is obtained by integrating die 3D 
ultrasound data in accordance with a direction, for instance the direction of orientation of the 

25 X-Ray acquisition means, as illustmted by the cylinder 52. Said volume rendered image 51 is 
used instead of the 2D ultrasound view 41 and combined with the 2D X-ray image 40 for 
generating a bunodal representation 53 in a similar way as previously desdribed. An 
advantage of tihis third approach is to provide a perspective view of the neighbourhood of die 
medical instrument 4, which are for instance the heart cavities. 

30 It should be noted that the generation means 14 could inversely gen^:ate a 

bimodal representation on the basis of the 3D ultrasound data set and replace X-ray 
information by ultrasound information. However, it is of less interest, because in this case, 
the bimodal repres^itation has an image field which is reduced to the one of the 3D 
ultrasound acquisition means. 
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The system is intended to provide both a live 2D X-Ray image and a live 3D 
ultrasound data set. Although a localization of the probe has already been perfomied in a 
peri- interventional step» it could be necessary to update such a localization ^en patient 
movements occur, in particular if the respiratory movements have to be compensated for. As 
5 a matt^ of fact, a patient movement may cause a change of position of the ultrasound probe 
within the X-Ray referential and therefore induce errors when mapping the 2D X-Ray data 
with the 3D ultrasound data. Therefore, in a fourth embodiment of the invention, the system 
conqprises controlling means for periodically triggering the means for localizing the 
ultrasoimd probe within the X-Ray referential. Such a triggering can be performed manually, 

10 when the user judges it necessary or automatically at regular time intervals, to the extent of 
retriggering this probe localization for every new 2D X-Ray image or 3D ultrasound data set 
acquired. It should be noted that in this case, the probe localization has to be performed in 
realrtime.. An advantage is that it avoids aay accumulation of errors when mapping the 2D 
X-Ray data with the 3D ultrasound data. 

15 In a fifth embodiment of the invention, the system comprises means for compensating 

any relative motion of the ultrasoxHid probe with respect to the patient heart in the interval 
between two successive probe localizations. These motion comqpensation means are intended 
to compensate a motion between a current 3D ultrasound data set 3DUS(tb+ 1) acquired at a . 
current time to+t and a previous 3D ultrasound data set 3DUS(tb) acquired at a previous time 

20 fo, both ultrasound data sets corresponding to a same phase of the heart cycle. Firstly, motion 
vectors linking points of the current 3D iiltrasound data set 3DUS(to+ 1) with points of the 
previous 3D ultrasoimd data set 3DUS(1b) are estimated and the points of the current 3D 
ultrasound data set 3DUS(tD+ 1) are displaced in accordance with the calculated motion 
vectors. Therefore, a motion compensated current 3D ultrasound data set MC(3DUS(to+t)) is 

25 obtained, which is expected to be closer to the previous 3D ultrasound data set 3DUS(to). In a 
first approach, a block- matching technique, well known to those skilled in the domain of 
video compression, is advantageously used. Referring to Fig. 10, which illustrates the 
principles of a block matching in the 2D case for simplicity reasons, the current and previous 
3D ultrasound data sets 3DUS(tb+t) and 3DUS(tO)are divided into blocks, for instance of 

30 8x8x8 points and for each block Bai of the current 3D ultrasound data set a search of the 

block Bno of the previous 3D ultrasound data set is perfojtmed, which leads to a Trifl vitti^im of 
conelatioiL A motion vector MV^Bn^Bn^ is obtained 

Therefore the motion compensation means are intmded to compensate in the current 
ultrasound image for small motions which may have occurred between time lb and time 1^ + 1 
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It should be noted that motion compensation is e£Qcdent when small dififerences exist 
between both 3D ulttasoimd data sets. An advantage of the fifth embodiment of the invention 
is to provide a solution for compensating errors of localization due to small movements of the 
probe with respect to the cavity to be explored, like the heart, which is compatible with live 
5 visualization. This solution is particularily advantageous when the probe localization cannot 
be performed in real-time, for instance when several angular views provided by the C-ann 
are involved. In this case, the motion condensation means may be advantageously used in 
tile mean time of two successive probe localizations. 

It should be noted that the system in accordance with the invention has a particular 

1 0 interest for electrophysiology procedures, which consist either in generating an electrical 

activation map of a heart cavity wall for diagnosing heart diseases or in burning a zone of the 
wall tissue, ^ich has been identified as abnormal. As a matter of fact, by providing a live 
visualisation of a large viewing field of the area of operation, in which Ifae medical 
instrument, the bone structures and the surrounding wall tissues are simultaneously visible, 

IS tiie system in accordance with the invention facilites a generation of the electrical activation 
map. 

The invention also relates to a method of guiding a medical instrument 4 in a 
patient body. Referring to Fig. 1 1, such a method comprises the steps of: 

- acquiring 60 a two-dimensional X-ray image of said medical instrument using an X-iay 
20 acquisition system, 

- acquiring 61athree-d]naensional ultrasound data set of said medical instrument using said 
ultrasound probe, 

-* localizing 62 said ultmsound probe in a referential of said X-ray acquisition system, 

- selecting 63 a region of interest of said medical instrument within said 3D ultrasound data 
25 set and providing a first localization of said region of interest within a referential of the 

ultrasound acquisition means, 

- converting 64 said first localization within said referential of the 3D ultrasound data set 
into a second X-Ray localization within said referential of the X-ray acquisition system, 

- generating 65 and displaying a bhnodal representation of said medical instrument in 

30 which both 2D X-ray image and the 3D ultrasound data mcluded in said region of interest 
are combined using said second localization. 
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The drawings and their description hereinbefore illustrate raflier than limit the 
invention. It will be evident that there are numerous alternatives, which fall within the scqpe 
of the appended claims. In this respect the following closing remadcs are made: there are 
numerous ways of implementing functions by means of items of hardware or software, or 

5 both. In this respect, the drawings are very diagrammatic, each representing only one possible 
embodiment of the invention. Thus, although a drawing shows different functions as different 
blocks, this by no means excludes that a single item of hardware or software carries out 
several functions, nor does it exclude that a single function is carried out by an assembly of 
items of hardware or software, or both. 

10 Any reference sign in a claim should not be construed as limiting the claim. Use of 

the verb '*to comprise** and its conjugations does not exclude the presence of elements or 
steps other than those stated in a claim. Use of the article "a'* or "an** preceding an element or 
step does not exclude the presence of a plurality of such elements or stqps. 
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CLAIMS 

1 . A medical system conq)rising: 

- a medical instrument to be guided in a patient body, 

S - X-Ray acquisition means for acquiring a two-dimensional X-ray image of said medical 
instrument, 

- ultrasound acquisition means for acquiring a three-dimensional ultrasound data set of said 
medical instrument using an ultrasound probe, 

- means for providing a localization of said ultrasound probe within a referential of said X- 
10 ray acquisition means, 

- means for selecting a region of interest around said medical instrument in the three- 
dimensional ultrasound data set, that deiine a first localization of said region of interest 
within a referential of said ultrasound acquisition means, 

- means for converting said first localization of said region of interest within said 

IS referential of the ultrasoxmd acquisition means into a second localization of said region of 

interest within said referential of the X-ray acquisition means, using said localization of 
the ultrasound probe, 

- means for generating and displaying a bi- modal representation of said medical instrument 
in which said two-dimensional X-ray image and the three-dimensional ultrasound data 

20 included in said region of interest are combined using said second localization. 

2. A system as claimed in claim 1, wh^ein said means for selecting a region of interest 
are intended to define a reference plane in which a part of said medical instrument is 
included. 



25 



A system as claimed in claim 2, wherein said region of interest is a 2D ultrasound 
image obtained by sampling said 3D ultrasound data set over said reference plane. 



4. 

30 



A system as claimed in claim 2, wherein said region of interest is obtained by 
cropping a 3D ultrasound data subset, which lies behind said reference plane or by 
cropping a slab which is formed around said refa:ence plane. 
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5. A system as claimed in claim 4, wherein said generating means aie intended to 

generate a volume rendered view of said region of interest within said 3D ultrasomid 
data set 

S 6. A system as claimed in claim 1, wherein said probe localization means aie intended to 
localize an active localizer, which has been arranged on said lillxasound probe. 

7. A system as claimed in claim 1, wherein said ultrasound probe is equq>ped with at 
least three non aligned and interdependent radio-opaque markers and said localization 

10 means are intended to localize said markers in at least a first 2D X-ray image having a 

first orientation angle in said referential. 

8. A system as claimed in claim 7» wherein said localization means are intended to 
further localize said markers in a second 2D X-ray image having a second orientation 

1 5 angle in said referential. 

9. A system as claimed in claim 1> wherein said selection means comprise means for 
detecting said medical instrument within said region of interest of the 3D ultrasound 
data set and said generating means ate intended to give to the points of the detected 

20 medical instrument in said bimodal representetion the X-ray intensity values of the 

corresponding points in the 2D X-Ray image. 

10. A system as claimed in claim 1, comprising means for segmenting a wall tissue 
region in the 3D ultrasound data set and said generating means are intended to give to 

25 the points belonging to sail wall tissue region the ultrasound intensity values of the 

corresponding points of said region of interest. 

1 1. A system as claimed in claim 1, wherein the X-Ray acquisition means are intended to 
provide live two-dimensional X-Ray images and the iiltrasound acquisition means 

30 live three-dimensional ultrasound data sets. 

12. A system as claimed in claim 11, comprising controlling means for periodically 
triggering the probe localization means. 



PHFR040033EPp 



18 

13. A system as claimed in claim 1 1, comprising means for compensating a motion 
between a current three-dimensional nltrasoimd data set acquired at a current time and 
a previous three>dim^ional xiltrasoimd data set acquired at a previous time. 

14. A method of guiding a medical instrument in a patient body, comprising the steps of: 

- acquiring a two-dimensional X-ray image of said medical instrument using an X-ray 
acquisition system, 

acquiring a three-dimensional \iltrasound data set of said medical instrument using said 
iiltrasound probe and an iiltrasomd acquisition system, 

- localizing said ultrasound probe in a referential of said X-ray acquisition system, 
selecting a region of interest of said medical instrument within said 3D ultrasoimd data 
set, that define a fii^ localization of said region of interest within a referential of said 
ultrasound acquisition system, 

converting said first localization within said referential of said ultrasound acquisition 
system into a second X-Ray localization within said referential of the X-ray acquisition 

system, 

generating and displaying a bimodal representation of said medical instrument in which 
said two-dimensional X-ray image and the three-dimensional ultrasound data included in 
said region of interest are combined using said second localization. 
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SYSTEM FOR GUIDING AMEDICAL INSTRUMENT IN A PATIENT BODY 
ABSTRACT 

The present invention relates to a medical system comprising a medical instrument to 
5 be guided in a patient body, means for acquiring a 2D X-ray image of said medical 

instrument, means for acquiring a 3D ultrasound data set of said medical instrument using an 
ultrasound probe, means for localizing said ultrasound probe in a referential of said X-zay 
acquisition means, means for selecting a region of interest around said medical instrument 
within the 3D ultrasound data set, that define a first localization of said region of interest in a 
10 referential of the ultrasound acquisition means, means for converting said first localization 
into a second localization in said referential of the X-ray acquisition means and means for 
generating a bknodal representation of said medical instmment detection by combining said 
2D X-ray image and the 3D ultrasound data included in said region of interest 
Ref,: Fig. 1 
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